A harmony processor is a device that is capable of creating one or more harmony signals that are pitch shifted versions of an input musical signal. Non-real-time harmony processors generally operate on pre-recorded audio signals that are typically file-based, and produce file-based output. Real-time harmony processors operate with fast processing with minimal look-ahead such that the output harmony voices are produced with a very short delay (less than 500 ms, and preferably less than 40 ms), making it practical for them to be used during a live performance. Typically, a real-time harmony processor will have either a microphone or instrument signal connected to the input, and will use a preset key, scale and scale-mode, or MIDI note information, to attempt to create musically correct harmonies. Some examples described in this disclosure are real-time harmony generators, although non-real-time harmony generators can also be provided.
There are a few real-time harmony processors currently available in the market. Although these products have been somewhat successful, there have always been three major complaints:                1. The harmonies generated from MIDI note information (generally referred to as chordal mode or vocoder mode) are unsatisfactory because they tend to change pitch much less frequently than the input melody signal. As a work-around, it is sometimes possible to enter each harmony note individually, or else create a custom chord to match each input melody note, but these are both difficult, tedious, and often require extra interaction from the performer in order to step through the notes. Also, because the changes in the harmony notes are not triggered by the melody notes themselves (instead either a foot switch or MIDI sequencer is commonly used), the harmonies can sound unnatural and out of step with the melody line.        2. When key and scale information is entered prior to starting a song, the harmonies can then be generated in step with the melody line (this is often called scalic mode). However, because the chord structure of a wide range of songs does not follow a set of rules that can be predetermined, the harmonies produced by this method often contain notes which are not musically correct because they are dissonant with respect to the accompaniment notes in situations where dissonance is unpleasant, thus limiting the usefulness of the harmony processing.        3. The existing products are very difficult to use because they require musical information such as key, scale, scale-mode, etc. to be entered before harmonies can be generated (for scalic mode), or else they require each harmony note or corresponding chord for each harmony note to be entered manually and then triggered throughout the performance.        
In order to overcome these shortcomings harmony generation systems that create musically correct harmonies for a wide range of songs, and do so without requiring any extra skill from the user beyond singing and playing his/her instrument are disclosed. Such systems and methods generally are based on:                1. Real-time extraction of individual notes from an instrument signal that contains multiple accompaniment notes mixed together (e.g., a strummed guitar). Note that this is a very different problem from recognizing chords from MIDI data which is currently quite common in the prior art. MIDI data is electronically generated (for example from an electronic keyboard) and contains all the individual note information explicitly, and note extraction is unnecessary.        2. Real-time computation of musically correct harmonies (generally consonant or deliberately dissonant) using a new method of harmony note generation that is not simply based on either the current recognized chord or the currently entered scale and scale-mode information for the song. Instead, harmonies are generated using a dynamic algorithm that looks at the current performance over several time-scales (current accompaniment notes, localized scale mode based on melody and accompaniment note history, and long-term dynamically-derived key, scale, and scale-mode information). As a result, the harmonies created move much more actively than existing chordal harmony methods, and are musically correct with respect to the accompaniment notes.        